Chaperones and Impersonators: Run-time Support for Contracts on Higher-Order, Stateful Values

Racket’s chaperone and impersonator language constructs provide run-time support for implementing higher-order contracts on mutable structures and abstract datatypes. Using chaperones and impersonators, contracts on mutable data can be enforced without changing the API to that data; contracts on large data structures can be checked lazily on only the accessed parts of the structure; contracts on objects and classes can be implemented with lower overhead; and contract wrappers can preserve object equality where appropriate. With this extension, Typed Racket, which relies on contracts for interoperation with untyped code, can now pass mutable values safely between typed and untyped modules. Although almost any proxy mechanism can support contracts, an unrestricted proxy layer would enable external code to violate the internal invariants of Racket libraries. Chaperones and impersonators are sufficiently expressive to implement contracts— allowing the remainder of the contract system to be implemented outside of the core run-time system—while preserving the abstraction mechanisms and reasoning that both Racket programmers and compiler analyses rely on. The paper includes a model of Racket with mutable data, chaperones, and impersonators, and proofs that the model maintains desired invariants. 1. Contracts in an Extensible Language An extensible progamming language like Racket enables thirdparty programmers to design and maintain seemingly core aspects of a programming language, such as a class system, a component system, or a type system. At the same time, extensibility creates constant pressure on the design of the core language to support new and different forms of extension. The core must evolve to support new forms of extension without exposing power to compromise abstractions and therefore inhibit composition. The Racket (Flatt and PLT 2010) contract system is a prime example of this trade-off in extensibility versus composition. The contract system can exist in its rich, state-of-the-art form largely because it can be implemented, modified, and deployed without requiring changes to the core run-time system and compiler. At the same time, since the contract system’s job is to help enforce invariants on functions and data, language extensions can accidentally subvert the intent of the contract system if the Racket core becomes too extensible or offers too much reflective capability. In this paper, we report on an addition to the Racket core that is driven by the need for a better and more extensible contract system, but where there exists an acute possibility of subverting core guarantees of the language. To properly enforce contracts on certain types of data, the contract system needs a mechanism for interposition on core operations—or intercession, in the terminology of the CLOS Metaobject Protocol (Kiczales et al. 1991)—so that use of those operations can trigger contract checks. For example, if a mutable vector has a contract on its elements, every use or modification of the vector should be guarded by a check. An up-front check does not suffice, because the vector may be modified concurrently or through a callback. Although the contract system needs interposition, the availability of unrestricted interposition could lead to more invariants broken than enforced. To balance the needs of extensibility and composition, we have developed a two-layer system of chaperones and impersonators. Chaperones and impersonators are both kinds of proxies, where a wrapper object interposes on operations intended for a target object. Chaperones can only constrain the behaviors of the objects that they wrap; for an interposed operation, a chaperone must either raise an exception, return the same value as the original object would have returned for the operation, or return a chaperone of the original object’s result. Impersonators, in contrast, are relatively free to change the apparent behavior of the objects that they wrap; while an impersonator usually behaves the same as the original object and returns the same values for interposed operations, the impersonator may behave differently or return different values. Overall, impersonators can be more expressive, but chaperones are allowed on more kinds of values. Together, chaperones and impersonators are powerful enough to implement an improved contract system without subverting guarantees that enable composition of language extensions. Thanks to chaperones and impersonators, the Racket contract system now supports higher-order contracts on mutable objects and abstract datatypes. This improvement directly benefits Racket programmers, and it benefits language extensions that are further layered on the contract system—notably Typed Racket (Tobin-Hochstadt and Felleisen 2008), which became more interoperable with untyped Racket as a result of the improvements. Last but not least, building just enough contract support into the core compiler and run-time system offers the promise of better performance, both for code that uses contracts and code that does not. Core-language support for interposition has already eliminated a factor of three slowdown for some object-oriented operations; this slowdown was present even if the program did not use contracts at all. Contract support for class-based objects now affects performance only for programs that use contracts. Section 2 motivates the conceptual need in contract checking for chaperones and impersonators. Section 3 describes the implementation of contracts in terms of chaperone and impersonator constructs that (unlike contracts) are built into the run-time system. Section 4 describes a formal model for a subset of Racket with chaperones and impersonators and proves that they preserve key invariants of the base language. Section 5 gives some performance numbers, and section 6 discusses related work. 2. Contracts in Racket We start with a review of predicate and function contracts in Racket, and then we explain how a generalization of the contract system to more kinds of data leads to new categories of contracts. 2.1 Predicates and Function Contracts A contract mediates the dynamic flow of a value across a boundary: In Racket, contracts most often mediate the boundaries between modules. For example, the left and right bubbles above could correspond to math.rkt and circles.rkt modules declared as